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Abstract 

This paper demonstrates fundamental limits of sensor networks for detection problems where the 
number of hypotheses is exponentially large. Such problems characterize many important applications 
including detection and classification of targets in a geographical area using a network of sensors, and 
detecting complex substances with a chemical sensor array. We refer to such applications as large- 
scale detection problems. Using the insight that these problems share fundamental similarities with the 
problem of communicating over a noisy channel, we define a quantity called the 'sensing capacity' 
and lower bound it for a number of sensor network models. The sensing capacity expression differs 
significantly from the channel capacity due to the fact that a fixed sensor configuration encodes all 
states of the environment. As a result, codewords are dependent and non-identically distributed. The 
sensing capacity provides a bound on the minimal number of sensors required to detect the state of an 
environment to within a desired accuracy. The results differ significantly from classical detection theory, 
and provide an intriguing connection between sensor networks and communications. In addition, we 
discuss the insight that sensing capacity provides for the problem of sensor selection. 
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I. Introduction 

A sensor network is deployed to obtain information about the state of an environment using 
multiple sensors. In many sensing applications, such as pollution monitoring and border security, 
the phenomena under observation has a large scale that exceeds the range of any one sensor. As a 
result, collecting measurements from multiple sensors is essential to the sensing task. Obtaining 
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information about an environment can be cast as either a 'detection' or an 'estimation' problem. 
In estimation problems such as the problem of estimating a continuous field to within a desired 
distortion, the state of the environment is continuous. In detection problems, such as binary 
hypothesis testing, the state of the environment is represented as a finite set of hypotheses. In 
this paper we study the problem of 'large-scale detection' where the state of the environment 
belongs to an exponentially large, structured set of hypotheses. Large-scale detection problems 
include many applications where a sensor network is deployed in order to monitor a large- 
scale phenomena. We exploit the structure of large-scale detection problems to demonstrate a 
fundamental information-theoretic relationship between the number of sensor measurements and 
ability of a sensor network to detect the state of the environment to within a desired accuracy. 

We obtain our results by drawing on an analogy between sensor networks and channel 
encoders. For a fixed sensor configuration, each state of the environment induces a corresponding 
set of sensor outputs. This set of sensor outputs can be viewed as a noise-corrupted 'codeword,' 
which must be 'decoded' in order to detect the state of the environment. Thus, the sensor network 
acts as a channel encoder. In order to motivate this perspective, we examine the following large- 
scale detection applications. 

Robotic mapping is the first large-scale application we consider [1]. In mapping, robots 
collect sensor measurements to map an unknown environment for the purpose of navigation. [2] 
introduced occupancy grids, one of the most popular approaches to this problem. In occupancy 
grids, the world is modeled as a discrete grid, where each grid location has a value corresponding 
to the state of the environment. For example, in a binary a grid a '0' can indicate free space 
while a '1' can indicate an obstacle. A robot traversing an unknown environment collects sensor 
measurements that encode the state of the environment. For example, a robot using a sonar 
sensor emits a wide acoustic pulse and measures the time until a reflected pulse is sensed. 
These readings are ambiguous, since one cannot infer the precise location of the obstacle that 
caused the reflection from a single sensor reading. In addition, sonar readings are noisy. As 
a result, multiple sensor measurements must be used to distinguish among an exponentially 
large number of possible grid states. The sequence of sonar readings can be viewed as a noise- 
corrupted codeword corresponding to the state of the grid. While robotic mapping systems have 
been successfully implemented in practice, little can be said about their theoretical performance. 
Theoretical understanding could shed light on the number of sensor measurements required 
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to map an unknown environment. In addition, theory can provide insight into questions about 
sensor selection. Is it better to use cheap, low power, wide angle sensors or expensive, high 
power, narrow angle sensors? A theoretical framework could provide general insight into such 
sensor selection questions. 

Video surveillance is another large scale detection problem. [3] used multi-camera sensor 
networks to detect and track objects across multiple areas, and [4] uses multiple cameras to 
localize moving objects in a room. The region under surveillance can be viewed as a three- 
dimensional grid. For example each grid position can have a binary value, representing motion 
or lack of motion in that grid position. As in the previous example, the number of states of this 
grid is exponential in the number of grid blocks. Each camera observes a subset of grid blocks, 
but introduces ambiguity by reducing a three-dimensional volume to a two-dimensional image. 
As a result multiple camera images must be combined to detect the state of the environment. 
The set of images encode the grid state. While practical systems for surveillance applications 
are deployed, a theoretical framework for understanding performance limits for such problems 
is not available. 

Identifying a complex chemical substance is a third example of a large-scale detection problem. 
In this application the output of chemical sensor arrays, consisting of heterogeneous chemical 
sensors, is used to distinguish among a large number of substances [5]. Each substance can be 
modeled as a mixture of constituent chemicals at various discrete concentration levels, resulting 
in an exponentially large set of possible states. Each chemical sensor in the array reacts to a 
subset of chemicals. For example, sensors can output a voltage proportional to a weighted sum 
of the concentrations of a subset of chemicals. The output of a chemical sensor array encodes 
the state of the sample being sensed. As in the previous two examples, theory could provide 
insight into the practical design of such sensor arrays. 

Target detection and classification in a geographical area is an important class of applications 
for sensor networks [6], and a final motivating example of a large-scale detection problem. We 
consider the problem of detection and classification using seismic sensors, as demonstrated in 
[6], [7]. The environment can be modeled as a discrete grid, where each position can contain 
targets of multiple types. The number of target configurations is exponential in the number of 
grid blocks. Seismic sensors are scattered randomly on this grid, and sense the vibrations of 
targets over subsets of the grid. The intensity of vibration is dependent on the target's distance 
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from the sensor, and therefore a single sensor cannot distinguish between many targets far away 
and a single target nearby. The set of seismic sensor outputs encode the location and class of 
targets in the field. 

All of the examples considered above share the following common elements. The state can 
be modeled as a discrete vector or grid, and the number of states is exponentially large. Sensors 
output noise-corrupted functions of subsets of the vector or grid. These sensor measurements 
must be fused in order to detect the state of the environment. In this paper we analyze the 
fundamental limits of this process by using the insight that the problem of large-scale detection 
and the problem of communicating over a noisy channel share essential similarities. 

II. Sensor Networks and Communication Channels 

The examples described in Section U motivate the sensor network model shown in Figure [Q 
A discrete target vector v represents the state of the environment. In this paper, the term 'state' 
and 'target vector' are used interchangeably. A fixed sensor configuration encodes the state as 
a vector of noiseless sensor outputs that form the codeword x. The observed, noisy sensor 
measurements are written as y, a noise-corrupted version of x. Finally, a detection algorithm 
uses y to compute a guess of the state of the environment v. 

The sensor model shown in Figure [T] is similar to the classical communication channel model 
shown in Figure |2l The target vector v corresponds to the message m being sent. The sensor 
network acts as a channel encoder, producing the codeword x. Finally, a detection algorithm acts 
as a channel decoder on the noise corrupted codeword y. Shannon's celebrated Channel capacity 
results provide limits for the communications channel [8]. Motivated by the similarity between 
the sensor network model and the communication channel model, we defined and bounded the 
sensing capacity in [9]. The sensing capacity plays a role in our sensor network model analogous 
to the role of channel capacity in a communications channel. However, because the models differ 
in significant ways, the notions of channel capacity and sensing capacity also differ. 

The most important difference between the sensor network model and a communication 
channel model is at the encoder. In communications, the content of the message and its codeword 
representation can be decoupled. Further, the channel encoder can implement any mapping 
between message and codeword. As a result, two highly similar messages can be differentiated 
with arbitrarily high accuracy. In contrast, a sensor network encoder uses the same sensor 
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configuration to encode all states of the environment. Further, since sensors react to some 
phenomena in the environment and are limited by physical constraints, the codeword associated 
with a particular state of the environment is a direct function of that state. Therefore the state and 
its codeword representation are coupled. As a result, two highly similar states of the environment 
cannot be distinguished with arbitrarily high accuracy. While similarities between the sensor 
network model and the channel model motivate the application of insights about communications 
from information theory, significant differences between the two models require care in applying 
such insights in order to understand the impact of these differences on the final theoretical results. 

Section [TIT] provides an overview of the main results presented in this paper, and reviews 
related work. Section [IV] presents sensing capacity results for non-spatial (e.g. chemical) sensing 
applications, while Section |V] demonstrates sensing capacity results for a sensor network model 
that accounts for spatial locality in sensor observations. Section [VI] concludes the paper and 
discusses future work. 

III. Main Results and Related Work 

We review the main theoretical results presented in this paper. In Section [IV] we introduce a 
simple but useful sensor network model that can be used to model sensing applications such as 
chemical sensing applications and computer network monitoring. For this model, we define and 
bound the sensing capacity. The sensing capacity bound differs significantly from the standard 
channel capacity results, and requires novel arguments to account for the constrained encoding 
of a sensor network. This is an important observation due to the use of mutual information as 
a sensor selection heuristic [10]. Our result shows that this is not the correct metric for large- 
scale detection applications. Extensions are presented to account for non-binary target vectors, 
target sparsity, and heterogeneous sensors. Plotting the sensing capacity bound, we demonstrate 
interesting sensing tradeoffs. For example, perhaps counter-intuitively, sensors of shorter range 
can achieve a desired detection accuracy with fewer measurements than sensors of longer range. 
Finally, we also compare our sensing capacity bound to simulated sensor network performance. 

In Section |V] we introduce a sensor network model that accounts for contiguity in a sensor's 
field of view. Contiguity is an essential aspect of many classes of sensors. For example, cameras 
observe localized regions and seismic sensors sense vibrations from nearby targets. We demon- 
strate sensing capacity bounds that account for such sensors by extending results about Markov 
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types [11], and use convex optimization to compute these bounds. The first result in Section 
IVl assumes the state of the environment is modeled as a one-dimensional vector. In Section 
IV-DI we extend this result to the case where the state of the environment is modeled as a two- 
dimensional grid. While a one-dimensional vector can model sensor network applications such 
as border security and traffic monitoring, results about two dimensions significantly increase the 
type of applications described by our models. 

The performance of sensor networks is limited by both sensing resources and non-sensing 
resources such as communications, computation, and power. One set of results has been obtained 
by considering the limitations that communications requirements impose on a sensor network. 
[12] extends the results in [13] to account for the different traffic models that arise in a sensor 
network. [14] studies network transport capacity for the case of regular sensor networks. [15] 
studies the impact of computational constraints and power on the communication efficiency 
of sensor networks. [16] has considered the interaction between transmission rates and power 
constraints. Another set of results has been obtained by extending results from compression 
to sensor networks. Distributed source coding [17], [18] provides limits on the compression of 
separately encoded correlated sources. [19] applies these results to sensor networks. [20] provides 
an overview of this area of research. 

In contrast to the work mentioned above, we focus directly on the limits of detecting the state 
of the environment using noisy sensor observations. The notion of sensing capacity characterizes 
the limits that sensing (e.g. sensor type, range, and noise) imposes on the attainable accuracy of 
detection. We do not examine the compression of sensor observations, or the resources required 
to communicate sensor observations to a point in the network. Instead, we focus on the limits of 
detection accuracy assuming complete availability of noisy sensor observations. Among existing 
work in information theory, the problem we investigate in this paper is unlike a source coding 
problem, and is similar to a channel coding problem. However, the sensor network model we 
investigate is fundamentally different than a standard communications channel. 

Our work is most closely related to work on detection and classification in sensor networks. 
[21] describes a large body of work on distributed detection where the number of hypotheses 
is small. [22], [23] extend this work to consider a decentralized binary detection problem with 
noisy communication links to obtain error exponents. [24] analyzes the performance of various 
classification schemes for classifying a Gaussian source. This is an m-ary problem where the 
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number of hypotheses is small. [25] analyzes the performance suboptimal classification schemes 
for classifying multiple targets. While the number of hypotheses is exponential in the number 
of targets, the large-scale detection problem of a large number of targets is not considered. [26] 
considers the problem of sensor placement for detecting the location of one or few targets in 
a grid. This problem is most closely related to the large-scale detection problems addressed in 
this paper. However, due to restrictions on the numbers of targets, the number of hypotheses 
remains small in comparison to a large-scale detection problem. A coding-based approach was 
used to bound the minimum number of sensors required for discrimination, and to propose 
structured sensor configurations. However, sensors were noiseless, and of limited type, and no 
notion of sensing capacity was considered. In contrast to existing to existing work on detection 
and classification in sensor networks, we demonstrate fundamental performance limits for large- 
scale detection problems. 

The problem of estimating a continuous field using a sensor network is an active area of 
research. [27] considers the relationship of transport capacity and the rate distortion function of 
a continuous random processes. [28] proves limits on the estimation of an inhomogeneous random 
fields using sensor that collect noisy point samples. Other work on the problem of estimating 
a continuous random field includes [29], [30], [31], [32]. [33] considers the estimation of 
continuous parameters of a set of underlying random processes through a noisy communications 
channel. The results presented in this paper consider the detection of a discrete state of an 
environment. We do not consider extensions to environments with a continuous state. 

IV. Sensing Capacity of the Arbitrary Connections Model 

In this section we define and analyze the sensing capacity of the arbitrary connections model, a 
simple but useful model introduced in [9]. We denote random variables and functions by upper- 
case letters, and instantiations or constants by lower-case letters. Bold-font denotes vectors. 
log(-) has base-2. Sets are denoted using calligraphic script. D(P\\Q) denotes the Kullback- 
Leibler distance and H(P) denotes entropy of a random variable with probability distribution P. 
H(Q\P) is the conditional entropy of a random variable with conditional probability distribution 
Q given another random variable with probability distribution P. 
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A. Arbitrary Connections Model 

Figure [3] shows an example of the arbitrary connections model. The state of the environment 
is modeled as a A;-dimensional binary target vector v. Each position in the vector may represent 
the presence of a target in an actual region in space, or may have other interpretations, such 
as the presence of a specific chemical in a sample. The possible target vectors are denoted v^ 
i E {1, . . . , 2 fe }. We say that 'a certain v has occurred' if that vector represents the true state. We 
define a sensor network s(k, n) as a graph showing the connections of n sensors to k positions 
in the target vector. The sensor network has n identical sensors. Sensor t senses exactly c out 
of the k spatial positions (shown in the graph as c connections). We refer to such sensors as 
having a range c. Ideally, each sensor produces a value x E X that is an arbitrary function of 
the targets which it senses, X£ = iff (via, ■ ■ ■ > v tt c )- Thus, the 'ideal output vector' of the sensor 
network x depends on the sensor connections, and on the target vector v that occurs. However, 
we assume that each sensor output y E y is corrupted by noise, so that the conditional p.m.f. 
Py\x(y\x) determines the output. Since the sensors are identical, Py\x is the same for all the 
sensors. Further, we assume that the noise is independent in the sensors, so that the 'sensor 
output vector' y relates to the ideal output x as P Y \x(y\ x ) — Y\ji=iPY\x(yi\xt)- Given the 
noise corrupted output y of the sensor network, we detect the target vector v which occurred 
by using a detector g(y). Because of the constrained encoding of a sensor network, we allow 
the decoder a distortion of D E [0, 1]. Denoting d^v^Vj) as the Hamming distance between 
two target vectors, the tolerable distortion region of Vi is V v . = {j : kdu(vi,Vj) < D}. Given 
that Vi occurred, the detector is in error if g(y) E" V Vi . 

Figure [3] shows the target vector v = (0,0,1,0,1,1,0) indicating 3 targets among the 7 
target positions. In this example, the sensing function iff is a sum that indicates the number of 
positions which contain a target, x e = J2u=i v a u , so that x E X — {0, 1, . . . , c}. Such a function 
could model a chemical sensor that is sensitive to a subset of chemicals and whose output is 
linearly proportional to the number of such chemicals present in the sample. More complex, e.g. 
nonlinear, relationships between chemicals and sensor output require a different choice of iff. In 
the figure,each sensor senses two target positions, and the sensors encode the target vector as 
x = (1, 0, 2, 1). However, due to noise, the observed vector of sensor outputs is y = (1, 1, 2, 1). 
The target vector v' = (0, 1, 1, 0, 1, 1, 0), which differs from v in one target position, is encoded 
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as x — (1, 1, 2, 1). As a result a detection algorithm can easily confuse v' for v, demonstrating 
the limitation imposed by the constrained encoding of a sensor network. 

The arbitrary connections model describes large-scale detection problems that do not have a 
spatial aspect. Examples of such applications include the detection of complex chemical and 
computer network monitoring. Disease detection in a population where individual sample can 
be combined is another such application. In addition to practical utility, this model is easy to 
analyze and provides useful insights into large-scale detection problems. 

B. Sensing Capacity Definitions 

How many sensor measurements must a sensor network collect to detect the a target vector to 
within a desired distortion? To answer this question we define the idea of a 'sensing capacity.' 
The probability of error of a sensor network given a that target vector Vi occurred is P e ^ s = 
Pr(error|i, s, x^ y) = Pr(g(y) ^ V v .\vi, s, Xi, y). The expected probability of error for a sensor 
network is P ejS = J2i Pe,i, s Pv{vi)- The rate R of a sensor network is defined as the ratio of target 
positions being sensed to the number of sensor measurements, R = ^. The sensing capacity of 
a sensor network, C(D), is defined as the maximum rate R such that below this rate there exists 
a sequence of sensor networks s(\nR],n) whose expected probability of error across all target 
vector goes to zero with increasing n, that is, P e s — > as n — > oo at a fixed rate R. 

Is C(D) nonzero? One of the main contributions of the theorem presented in this section is 
to demonstrate that the sensing capacity can be strictly positive for the arbitrary connections 
model. We use a random coding argument to obtain a bound on the sensing capacity for the 
arbitrary connections model. Instead of constructing a sequence of sensor network directly, we 
bound the expected probability of error, averaged over a randomly generated ensemble of sensor 
networks. The sensor networks are generated as follows. Each sensor connects to c randomly 
chosen target positions out of the k possible positions. The connections are made independently, 
and are chosen with replacement. Therefore a sensor can choose the same target position more 
than once. When we take the expectation over all such randomly generated sensor networks, the 
ideal sensor outputs associated with each target vector become random. Since a sensor network 
produces a codeword that is a function of the target vector, codeword distribution depends on the 
occurring target vector. We denote the random vector which occurs when Vi is the target vector 
as Xi. Because each sensor forms its connections independently, Pjjq (xi) = n™=i PxAxu). It is 
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important to note that sensor outputs are in general not independent, and are only independent 
when we condition on the occurrence of a particular target vector. Further, it is important to 
note that the random vectors Xi and Xj, associated with a pair of target vectors, Vi and ir- 
respectively, are not independent, since the sensor connections produce a dependency between 
them. Thus, the 'codewords' {Xi,i = 1,2, . . . ,2 k } of the sensor network are non-identical 
and dependent on each other, unlike channel codes in classical information theory. Using this 
probabilistic model for sensor network generation, we write the expected probability of error, 
averaged over the sensor network ensemble as P e = E s[P e ,s]- Theorem[T]in Section ITV-Cl bounds 
this quantity to prove a lower bound Clb(D) on the sensing capacity C(D). 

The statement of the result presented in this section relies on the method of types [11], and 
requires an explanation of types and joint types. Since in the random sensor network construction 
each sensor connects to c target positions independently, the distribution of a sensor's ideal output 
Xi depends only on the type 7 = (70,71) of Vi. The type of target vector Vi is a histogram 
of the number of 0's and l's in v^,. Here, 70 denotes the fraction of zeros in t>$. Since sensor 
connections are generated independently and uniformly across target positions in the arbitrary 
connections model we can write, Px^Xi) = P 1,n (xi) = YYe=i P^^u) f° r all v i °f the same 
type 7. 

Since a single sensor network encodes all target vectors, pairs of codewords are dependent, 
unlike codes in communications. The joint probability of two codewords PxiXj depends on the 
joint type of the target vectors Vi and Vj. The joint type is A = (A o, Aqi, Ai , An). Here, A i is 
the fraction of positions in Vi, Vj where Vi has bit '0' while Vj has bit '1'. Similarly, we define 

Aoo, A10, An. 

Following the notation introduced in [11], A G Vk({0, l} 2 ), indicating that A is in the set of 
joint types of fc-bit binary vector pairs. Again, since sensor connections are generated indepen- 
dently and with uniform probability across target positions, Pxi,Xj{xi,Xj) = P x ' n (xi, Xj) = 
Tl"=i P X ( x iz-> x jt) f° r a ^ v ii v j °f me same joint type A. Since the joint type A also defines the 
type 7 of vi, we have A 00 + A i = 70, Ai + An = 71. 

We give specific examples of these quantities using the example shown in Figure [3] where 
c = 2 and sensors count the number of targets present in the target positions that they sense. 
Table U lists the types of four vectors Vj, and their joint type with the target vector in the example 
Vi = 0010110. Given a target vector, a sensor will output '2' only if both of its connections 
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connect to positions with a '1.' For a vector of type 7, this occurs with probability (7 X ) 2 . Table 
|n] describes the complete output p.m.f. for a randomly generated sensor, given that a vector of 
type 7 occurred. Given two target vectors Vi, Vj of joint type A, a sensor will output '0' for 
both target vectors only if both its connections are connected to target positions that have a '0' 
bit in both these target vectors. This happens with probability (A o) 2 - Table Hill lists the complete 
joint p.m.f. PxiXj(xi,Xj) = P x (x i) Xj) of a randomly generated sensor for two target vectors 
with a joint type A. 

C Sensing Capacity Lower Bound 

We specify two probability distributions which we will utilize in the main theorem of this 
section. The first is the joint distribution of the ideal output Xi when Vi occurs, and the noise 
corrupted output y, P Xi Y(xi, y) = [ILi PxiY&ie, Vi) = ECU Pxi(xu)Py\x(vt\xu)- Th e second 
distribution is the joint distribution of the ideal output Xi corresponding to Vi and the noise 
corrupted output y generated by the occurrence of a different target vector Vj. We can write 
this joint distribution as Q% iY (xi,y) = Y\^ =1 Q^ Y { x it»yt) = Yll=i E ae * Px^x^it, x A = 
a)PY\x(yi\xj = a). Note that although Y was produced by Xj, Xi and Y are dependent 
because of the dependence of Xi and Xj. 

We argued earlier that due to the random sensor network construction, P Xi and PxiXj can 
be compute using the type 7 of Vi and joint type A of Vj respectively. Thus, we write 
PxM x iiV) — EL=i PxiY^itiVi) wnere PxiY^iiV) ~ P^^^Pyixiylxi). Similarly, we write 

QxM^v) = nt=iQxiY{ x a,yi) where QxiY^hV) = T,aex pX ( x i> x i = a ) p Y\x(y\xj = a). 
We can now ready to state the main theorem of this section. 

Theorem 1 (Sensing Capacity Theorem for the Arbitrary Connections Model): The sensing ca- 
pacity at distortion D is bounded as, 

C (D) >_C LB(D) = m ,„ ^'gj (D 

A i+Aio>D v ' y " 

Aoo+Aoi=7o 

Aio+An=7i 

where 7 = (0.5, 0.5) and A = (A o, A i, Ai , An) is an arbitrary probability mass functions. 

The most striking difference between the result shown in Theorem [fl and Shannon's channel 
capacity results is that the bound on the sensing capacity is not a mutual information. From 
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the definition of Q^.y, we notice that if the 'codewords' Xi were independent, the Kullback- 
Leibler distance would reduce to the mutual information between Xi and its noisy version Y. 
This is an important difference because of the frequent use of mutual information as a sensor 
selection metric (e.g. [10]), and indicates that the mutual information is not the correct notion of 
information for large-scale detection applications. The difference between channel capacity and 
sensing capacity arises due to different codeword geometries. In proofs of the achievability of 
channel capacity, since a codeword can be arbitrarily assigned to a message in communications, 
codewords are distributed uniformly. In a sensor network, the codeword distribution depends 
on the state of the environment (the target vector). Codewords are clustered, with similar target 
vectors encoded as similar codewords. As a result, similar target vectors are more likely to be 
confused due to noise than dissimilar target vectors. The Kullback-Leibler distance in Theorem 
Q] is the appropriate information measure for such a codeword geometry. The denominator in 
Theorem (0Q) accounts for disparities in the size of codewords clusters. The minimization over 
the joint type appears because the "closest" target vectors dominate the error probability. Thus, 
the sensing capacity is similar to classical channel capacity, with differences arising due to the 
non-identical, dependent codeword distribution. 

The proof of Theorem Q] broadly follows the proof of channel capacity provided by Gallager 
[34], by analyzing a union bound of pair- wise error probabilities, averaged over randomly 
generated sensor networks. However, it differs from [34] in several important ways. In our 
sensor network model, the codewords are dependent on each and non-identically distributed. To 
prove our bound, we group the exponential number of pair-wise error terms into a polynomial 
number of terms using the method of types. 
Proof: 

We assume a maximum-likelihood detector #ml(j/) = argmax- P Y \x(y\ x j)- For this detector, 
we consider P e = £V Pe,u where we assume that the target vectors are equally likely. P e;i 
is the error probability when the i l h target vector occurs, averaged over all randomly generated 
sensor networks. For a fixed sensor network s there is a known and fixed correspondence between 
target vectors Vi and codewords Since our sensor network is chosen randomly, the set of 
codewords is random, C = {X 1 , . . . , X 2 k}. 

P e = E VYC [Pr(g(Y) #V V \V,C,Y)] (2) 
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Using the fact that we are taking the expectation of a probability, we bound P e as follows, 



P P < E 



VYC 



J2Hg(Y)es w \v,c,Yy 



(3) 



where p £ [0, 1], and {S\,S2, . . .} is a partition of the complement of Dy, denoted Vy. Using 
the union bound, we upper bound the probability Pr(g(Y) £ S W \V,C, Y) as follows, 



Pe <E 



VYC 



(4) 



J2[J2 F <9(Y)=j\V,C,Y 

VJ \j£S w 

The term Pr(g(Y) = j\V,C,Y) is a pairwise error term that depends only on the codewords 
Xi and Xj. Using this observation, the fact that x p is a concave function for p £ [0, 1], and 
Jensen's inequality, we obtain, 



P P < E 



VYXi 



(5) 



The term Pr(g(Y) = Vj\V, Xi, Xj,Y) is a one zero function, equaling one when g(Y) = 
Vj and zero otherwise. Using our assumption that g is an ML detector we upper bound this 
probability as follows, 



PY\x{y\xj 



PY\x{y\xi) 



1+p 



(6) 



E E E p Xi\XiMxi 

The bound in equation © has an exponentially large number of terms. Earlier in this paper, it 
was shown that the distributions in this bound can be completely specified by the type 7 and 
joint type A rather than the specific i,j pair of target vectors. To do this, we choose each S w to 
be a distinct joint type A, and let w index the set S^(D) of all A that are the joint type of Vi 
and Vj £ X>£. We group the summation over i according to the type of Vi. Grouping according 
to the type and joint type enables us to take advantage of the fact that the number of types is 
polynomial in k. After grouping according to types, we write equation © as, 



p e <^E«(^ fc ) E E p7,n (^) p ^ ( 



y\Xi 



Xi£X n yey n 



E \P(\k) E P^Mxi) 
\eS-f(D) 



PY\x{y\xj) 
Py\x{y\xi) 



i+p 



(7) 
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where a(y, k) is the number of target vectors Vi of length k and type 7, and where /?(A, k) is 
the number of target vectors Vj of length k and joint type A with a target vector Vi of type 7. 
S~f(D) is defined as 

Syp) = {A : A01 + A10 > D, Aoo + A i = 7q, A10 + A n = 7J (8) 

Using standard results from the method of types [11] about the number of binary vectors of a 
given type, we obtain the bound, a (7, fc) < 2 kH<a \ The number of vectors with a given joint 
type is bounded as, 

/3(A, jfe) = f *7° ^ f ^ ^ < 2*( ff W- ff W) (9) 



\k\ 00 J \k\iiy 

Combining equation © with the bounds on a and /5, and using the conditional independence 
of sensor outputs, we obtain, 

P e <^2 ^ 2- k{1 ~ H ^ ) h kp{H{x) - Hi ' l)) 2- nEip ' X) (10) 
1 Aes T (i>) 

where E(p, A) is defined as below, 

E( P ,x) = -log ( J2 X^K^mC&k) 1 ^ E pX ( a 3\ a i) p Y\x( b h)^)} ( n ) 

Since the number of types 7 and joint types A are upper bounded by (k + l) 2 and [k + l) 4 
respectively, and k = \nR~\ , implying k < nR + 1, (fTOl) is bounded as, 

P e < 2~ n(oi ( n )~ KBr(7? '- D )) (12) 

where oi(n) — >• as n — ► 00, and where E r (R, D) is defined as, 

£ r (P,L>) = min min max A) + R(l — HOy)) - pR(H(X) - H(y))) (13) 

7 AeS-y(D) 0<p<l 

The average error probability P e — ► as n — > 00 if E r (R, D) > 0. Observing that E(0,X) = 
VA, we let p go to zero, rather than optimizing it, thus resulting in a lower bound on E r (R, D). 
In the above expression, this implies that in order for R to be achievable E(j>,x ^ + R l ~ H h) _ 
R(H(X) — H(y)) must be positive for all types and joint types as p — ► 0. 

For H(y) 7^ 1, — > 00 as p — > 0. For such a 7, P e — > since E r (R, D) is positive for 

all rates P. Since we seek to bound i? for which E r (R, D) is positive for all types and joint 
types, we let 7 = (0.5, 0.5). This implies that as p — > 0, i? is achievable when the derivative of 
E(p, A) with respect to p at p = is greater than R(H(X) — H(y)). It can be easily shown that, 
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dE(p, A)/<9p|^_ = D(Px iY \\Qx iY )- Using this derivative in the analysis above, we see that the 
achievable rates R are bounded as below. 

„, . D(PIyWQx,y) 

^00+^01=70 

Aio+Aii=7i 

where 7 = (0.5, 0.5), and A is an arbitrary p.m.f. since n — > 00. Therefore, the right hand side 
of (fl~4l) is a lower bound on C(D). ■ 



D. Numerical Results 

We compute the capacity bound Clb(D) in (Q~|) for various distortions, noise levels, and sensor 
ranges. A sensor of range c is connected to c target positions. We assume that the sensing function 
^ simply counts the number of target positions in the sensor range with a target present. The 
sensor noise model assumes that the probability of counting error decays exponentially with 
the error magnitude. In the figures, 'Noise = p' indicates that for a sensor, P(Y ^ X) = p, 
with y = X assumed. In Figure HI we demonstrate Clb(D) for various sensor noise levels and 
ranges. We compute this bound by systematically sampling the space of possible A. While A is a 
four-dimensional vector, because of constraints we need to sample only two dimensions in order 
search over all valid A. In all cases, C LB (D) approaches as D approaches 0. This occurs because 
similar target vectors have similar codewords due to dependence in the codeword distribution. 
The relative magnitude of the bounds for sensors of various c and noise levels describes tradeoffs 
among sensor types that can be captured by our result. Some tradeoffs are intuitive. For example, 
lower noise sensor of range c have a higher sensing capacity than higher noise sensors of the 
same range. Other tradeoffs are more complex. For example the tradeoff between shorter and 
longer range sensors depends on the desired distortion. Sensors of range 4 and noise 0.10 result 
in a higher sensing capacity than sensors of range 2 and noise 0.01 for distortion above 0.047. 
The opposite is true for distortions below 0.047. Thus, the bound presented in © describes a 
complex tradeoffs between sensor noise, sensor range, and the desired detection accuracy. 

Figure |5] shows Clb(D) at D = 0.1 as a function of sensor noise level for sensors of 
various range and sensing functions. This figure demonstrates that the strategy of simple sensor 
replication, which is a popular practical method for reducing error probability, can be inefficient. 
For example, for sensors of range 4 and a sum sensing function, a rate of 0.61 is achievable 
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at noise level 0.1. If each sensor with noise 0.1 is replicated three times and majority decoding 
is used, the noise can be reduced to 3 x (0.1) 2 x 0.9 + (0.1) 3 = 0.028. For a noise level of 
0.028, Clb(O.I) equals 0.91 for a sensor of range 4 and a sum sensing function. However, due 
to sensor replication, the rate is reduced to 0.91/3 = 0.303. This rate is significantly lower 
than the rate of 0.61 for sensors of noise 0.1 achievable by using our random sensor network 
construction. Thus, the bound indicates that cooperative sensor strategies can require significantly 
fewer sensor measurements than sensor replication. Figure [5] also shows Clb(D) at D = 0.1 
for sensors with c = 4 and a weighted sum sensing function with weights {1,0.5,0.25,0.1}. 
This sensing function has a higher sensing capacity than sensors with the same range and an 
un-weighted sum sensing function across all noise levels. We conjecture that this occurs because 
a weighted sum can distinguish among more target configurations than an un-weighted sum. 
Interestingly, the gap between the two sensing functions increases with increasing noise. 

Using the loopy belief propagation algorithm [35] we empirically examined sensor probability 
of error as a function of rate. We generated sensor networks of various rates by setting the number 
of targets, and varying the number of sensors. We chose the number of connections to be c = 4, 
the distortion level to be 0.1, and the noise level to be 0.1 (i.e. P(Y ^ X) = 0.1, with y = X). 
As in the previous section, we assume that the probability of error decays exponentially with 
error magnitude. We empirically evaluated the average error rate obtained in decoding target 
vectors in a randomly generated set of sensor networks. We plotted the average error rate for 
each rate value, and for various numbers of targets as shown in Figure [6] As the number of 
targets increase, the transition from high error to low error rate becomes increasingly sharp. 
However, all the error curves are well below the capacity value Cxs(O.l) = 0.62. We conjecture 
that this occurs because belief propagation is suboptimal for graphs with cycles. 

E. Extensions 

Section IIV-AI introduced a sensor network model where each sensor is allowed to make 
arbitrary connections to the target vector. In several situations, more complex sensor network 
models may be necessary. This section describes extensions of the arbitrary connection model. 
Extensions that account for contiguity in sensor connections require a new model and are 
discussed in Section |V] The first extension considers non-binary target vectors. Binary target 
vectors indicate the presence or absence of targets at the spatial positions. A target vector over a 
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general finite alphabet may indicate, in addition to the presence of targets, the class of a target. 
Alternatively, the entries of non-binary vectors can indicate levels of intensity or concentration. 
Assuming a non-binary target vector, we can define types and joint types over an alphabet V, 
and apply the same analysis as before to obtain the sensing capacity bound below. 

C(D) > C LB (D) = min ^gfM (15) 

J2a^b X ab>D 
X)& A ai) =7a 

where 7 = ^7 a = y^y, a G Vj, while A = (\ a b, a,b G V) is an arbitrary probability mass 
function. 

The second extension allows the following a priori distribution over target vectors. Assume 
that each target position is generated i.i.d. with probability Py over the alphabet V. This may 
model the fact that targets are sparsely present. The previous analysis can be extended to a 
Maximum- a-Posteriori (MAP) detector, instead of the ML detector considered earlier, resulting 
in the following sensing capacity bound. 

D (PxyWQx y) 

cm > cmd) = T <«> 

X]b X ab = 1ia 

where 7^ = Py, A = (\ a b, a, b G V) is an arbitrary probability mass function and 7^ is the 
marginal of A calculated as •jjb = J2 a ^ab- 

A third extension accounts for heterogenous sensors, where each class of sensor possibly has 
a different range c, noise model Py\x, and/or sensing function ^. Let the sensor of class I be 
used with a given relative frequency a\. For such a model the sensing capacity bound is as 
follows. 

C(D)>C LB (D)= min — V / (17) 

l^a^b X ab>IJ 

where 7, = Py, A = (X a b, a, b & V) is an arbitrary probability mass function and ~fj is the 
marginal of A calculated as 7^ = J2 a ^ab- 



V. Sensing Capacity of Contiguous Connections Model 

In this section, we analyze the sensing capacity of a sensor network model that models 
contiguity in a sensor's connections. Figure [7] shows an example of such a model. Sensor £ 
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is connected to exactly c contiguous positions out of the k spatial positions. In contrast, the 
arbitrary connections model analyzed in the previous section did not account for localized sensor 
observations since each sensor could sense any c (not necessarily contiguous) spatial positions. 

A. Higher Order Types 

The statement of the result for contiguous models requires higher order types [11]. We 
introduce circular c-order types and circular c-order joint types. We define the circular c-order 
type of a binary sequence (i.e. a target vector) as a 2 C dimensional vector, 7, where each entry 
in the vector corresponds to the frequency of occurrence of one of the possible subsequences 
of length c. A circular sequence is one in which the last element of the sequence precedes 
the first element of the sequence. The total number of subsequences of length c that can occur 
in a circular sequence of length k is k. For example, for a binary target vector and c = 2, 
7 = (7oo,7oi,7io,7n)- While it is possible to prove our bound using non-circular types as 
shown in [36], circular types lead to the same asymptotic result with the benefit of significantly 
simpler notation. The notational simplicity arises out of the fact that the lower order circular 
types are precise marginals of the higher order circular types. Although all the types in this 
section are circular, we will omit the word 'circular' when referring to types in the remainder 
of this section for brevity. 

We denote the set of all c-order types over the alphabet V c for target vectors of length k as 
Vk(V c ). Since each sensor independently chooses a block of c contiguous spatial positions, the 
distribution of its ideal output Xj depends only on the c-order type 7 of the target vector Vi 
which occurs. For a sensing function ^ and a target vector Vi of type 7, 

P Xi (X l = x)= 7a = P 7 (x) (18) 

aGV c 

Next, we note that the joint distribution -Pjsqx, depends on the c-order joint type X of the 
i th and j th target vectors Vi,Vj. A is the vector of A( )( b ), the fraction of positions in v^, Vj 
where Vi has a bit subsequence a while Vj has a bit subsequence b. For example, when c = 2 
and V = {0, 1}, A = (A( o)(oo), • • • , A(n)(n)). We denote the set of all c-order joint types over 
the alphabet V c for target vectors of length k as Vk(V c , V c ). Each A e Vk(V c , V c ) must satisfy 
the normalization constraint that the sum over all entries of A equals one. Since the joint type 
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A also defines the type 7 of Vi, for all {a} 6 V c we must have -y a = J2b&v c \a)(b)- Taking 
advantage of the fact that for circular types, lower order types are precise marginals of higher 
order types, we denote A (a ) (6 ) = Ea'ev-i J2b'ev^ \aa>)(bb')- \a)(b) is the normalized count of 
locations where target vector i has value a while target vector j has value b. Since each sensor 
depends only on the c contiguous targets bits which it senses, Pxi.Xj depends only on the joint 
type A. For target vectors Vi,Vj of c-order joint type A, 

Px^Xi = Xi,Xj = Xj) = ^ \a)(b) = P X (x i ,x j ) (19) 

a,6GV c 
<B(a)=Xi, *(6)=Xj 

For example, for binary target vectors and c = 2, vectors 00000000, 01101000, 01000111 have 
7 = (1,0,0,0), (3/8,2/8,2/8, 1/8), (2/8,2/8,2/8,2/8) respectively. Table [V] contains the 2- 
order joint type of two target vectors. Consider a sensor network where each sensor is randomly 
connected to c = 2 contiguous spatial positions. We assume that ^ outputs the number of 
targets which the sensor observes. Thus, each sensor has an ideal output alphabet X = {0, 1, 2}. 
For target vectors of type 7, P(Xi = 0) = 700, P{X t = 1) = 701 + 710, P{Xi = 2) = 711 
respectively. Given two target vectors Vi,Vj of joint type A, a sensor will output '0' for both 
target vectors only if both of its connections see a '0' bit in both target vectors. This happens 
with probability A( o)(oo)- Table |V] lists the joint p.m.f. P Xi Xj(xi,Xj) = P x (xi,Xj) for all output 
pairs Xi,Xj corresponding to joint type A. The table shows that X^Xj are not independent, in 
general. 

To prove Theorem [Tj we bounded the number of target vectors Vj that have a given joint type 
with a target vector Vi in equation ©. To prove a sensing capacity bound for the contiguous 
connections model we prove a bound on the number of target vectors Vj that have a joint c- 
order type A with a target vector of c-order type 7 in the lemma below. Before proceeding, we 
introduce the following notation. The set of length k target vectors of c-order type 7 is denoted 
Tjf. The set of pairs of length k target vectors of joint type A is denoted T£. The set of length 
k target vectors that have joint c-order type A with a given vector of type 7, is denoted 

Lemma 1 (Bound on |^L| The number of binary vectors of length k with c-order joint 
type A for a given vector of c-order type 7, denoted |^JU, is bounded as follows 

\Tt J < C (k)2 k ^' x ^- H ^ (20) 
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C(k) = 2 2 ^k 2C '\k + lf {c x) and A' = {A (a){6) , Va, b e V^ 1 } is a probability mass function 
defined as A' (a)(b) = J2 a ,bev A(aa)(w>)- A = {A( oa )( 66) , Vo, 6 G V c ~\Va,& € V} is a conditional 
probability mass function defined as \(aa)(bb) = X [ aa)(bb) . 7' = {^' a ,\/a e V c_1 } is probability 
mass function defined as 7^ = Xlaev7aa- 7 = {7aa,Va e V c_1 ,Va G V} is a conditional 
probability mass function defined as 7 aa = 

Proof: To bound | / 7^L|, we begin by bounding |Z^|. The c-order type 7 of a binary vector 
specifies a 2-order type (referred to as a Markov type) of a vector who entries are in an alphabet 
of cardinality 2 C_1 . Consider the vector 011010001. Denoting a pair of bits using {0, 1, 2, 3}, as 
we move from left to right over this vector, one bit at a time, the sequence obtained is 132120012. 
The 3-order type 7 specifies the 2-order type over this new vector. For example, the fraction 
of times 1 transitions to 3 is equal to 7011 and the fraction of times 1 transitions to 2 is equal 
to 7oio- Any c-order type over a binary sequence can thus be mapped to a 2-order type over a 
sequence with symbols in an alphabet of cardinality 2 C_1 . [37] proves bounds on the number 
of sequences that correspond to a 2-order circular type over a sequence with an alphabet V. 
Given our mapping from a c-order type to a 2-order type, we can apply this result to obtain the 
following bound, 

\TJ;\ > Ci(ib)2 fc ( H W- H ^)) = d(£;)2 feif ^') (21) 

where d(k) = k~ 2 ° (k+ 1)~ 2 lc . We now bound \T£\ using a similar argument. The c-order 
joint type A of a pair of binary vectors specifies a 2-order type of a single vector whose entries 
are symbols from an alphabet of cardinality 2 2< - c ~ 1 \ We consider an example for a 3-order joint 
type, and for vectors v = 011010001 and v' = 101011011. We can rewrite these vectors as a 
single vector whose entries at location % are defined by the pair of entries v i: v[ and the subsequent 
pair of entries Vi+i, v' i+1 . These four entries, combined as V{V j + it>X'+i are mapped to a symbol in 
an alphabet of cardinality 2 4 by reading the entries as a binary number (i.e. 0000 = 0, 0001 = 1, 
. . .). In this manner, v and v' are mapped to a vector (6, 14, 10, 9, 11, 2, 1, 7, 11). The 3-order 
joint type A specifies the 2-order type over this new vector. For example, the fraction of times 
1 transitions to 7 is equal to A(ooi)(on) an d the fraction of times 2 transitions to 1 is equal to 
A(ooo)(ioi)- Any c-order joint type over a binary sequence can thus be mapped to a 2-order type 
over a sequence with symbols in an alphabet of cardinality 2 2<c ~ 1 \ We use the results of [37] 
again. Given our mapping from a c-order joint type to a 2-order type, we can apply this result 
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to obtain the following bound, 

|T A fc | < c 2 2 k(H W- H ^')) = c 2 2 kH ^ (22) 
where C 2 = 2 2(c ~ 1 ). We observe that I'T^U depends only on the type 7 of the vector on which 

, \T k \ 

we are conditioning, and not on the actual vector. Therefore, \Tx\y\ = j^j- Using equations (|2lj) 
and d22l) . we obtain the following bound, 

\T k h \ < C (k)2 k ^' x ^- H ^ (23) 

where C{k) = C^\k)C 2 



B. Sensing Capacity Lower Bound 

We define -P^.y Qxy as me y were defined for the arbitrary connections model bounds, with 
the only difference arising to the use of c-order types instead of types. 

Theorem 2 (Sensing Capacity Theorem for the Contiguous Connections Model): The sensing 
capacity at distortion D satisfies, 

D (PZyWQxy) 

C{D)>C LB {D)= min R r , (24) 

where A G P({0, 1} C , {0, 1} C ), la = E be{ o,i}c \«)<p), and H(j\ 7 ') = 1. 

If we specialize this result to the case of c = 1, this theorem provides a bound that coincides 
with our bound for the arbitrary connections model. The proof of the sensing capacity lower 
bound is similar for the arbitrary and contiguous connections models. The main differences in 
the proofs arise due to the contiguity of sensor field of view, which necessitates the use of 
c-order types. Extensions demonstrated in Section HV-EI for the arbitrary connections model can 
be easily applied to the contiguous connections model. 

Proof Outline: The proof of Theorem [2] is essentially identical to the proof of Theorem \T\ 
with types and joint types replaced by c-order types and joint types. The use of these higher 
order types requires counting arguments described in Lemma [T] For c-order types, we bound a 
in equation © as follows, 

a(7,fc) = \T k \ < 2 kH{ ^' ) (25) 
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For c-order joint types, we bound f3(\,k) = |2^L| in equation © using Lemma [Q The set 
S~f(D) is defined as, 



k 6G{0,1} C ) 

Given these new bounds and definitions, and the substitution of c-order types for types, the proof 
of Theorem Q] can be applied directly to prove Theorem [2l 

C. Numerical Results 

In Figure [H we compare Clb(D = 0.025) for sensor networks with localized (i.e. contiguous 
connections model) and non-localized (i.e. arbitrary connections model) sensing. We assume that 
the sensing function ^ is a weighted additive function, with weights {1, 0.5, 0.25, 0.1} for c = 4 
and {1, 0.5, 0.25} for c = 3. The sensor noise model used throughout this section assumes that 
the probability of error decays exponentially with the error magnitude. In the figures, 'Noise = 
p' indicates that for a sensor, P(Y ^ X) = p, with y = X assumed. Contiguous sensor field of 
view causes a significant reduction in sensing capacity. We conjecture that this effect is similar to 
the inferior performance of channel codes that have finite memory, such as convolutional codes, 
as opposed to LDPC codes. Further, it is interesting to note that the gap in sensing capacity 
between sensors of range c = 3 and c = 4 is larger for the arbitrary connections model than the 
contiguous connections model. 

To compute the bound shown in Theorem [2l we solve a sequence of convex optimization 
problems. Rather than computing the bound directly, we find the largest R for which the 
minimum of /(A) = D {P^yWQ^y) ~ R(H(X\X') - H(y\y')) over all valid A is greater 
than 0. Minimizing /(A) is a convex optimization problem since /(A) is convex in A and the 
set of valid A is convex. Since H( ; y\j') = 1, the convexity of /(A) in A can be proven using 
the log- sum inequality and the concavity of entropy. 

D. Extension to Two-dimensional Fields 

The sensing capacity bounds obtained in this section can be extended from discrete target 
vectors to two dimensional 'target fields.' This extension requires the introduction of two di- 
mensional types. Such types are histograms over the set of possible two dimensional patterns. 




(26) 
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We first analyzed the sensing capacity for a two-dimensional contiguous connections model in 
[38]. 

Figure |9] shows an example of our sensor network model. The state of the environment is 
modeled as a k x k grid with k 2 spatial positions. Each discrete position may contain no target 
or one target, and therefore the target configuration is represented by a A; 2 -bit target field /. The 
possible target fields are denoted fi, i E {1, . . . , 2 fc2 }. Target fields occur with equal probability. 
The sensor network has n identical sensors. Sensor I located at grid block F h senses a set 
of contiguous target positions within a Euclidean distance c of its grid location (though this 
approach can be extended to other sensor coverage models). Circular boundary conditions are 
assumed. Figure [9] depicts sensors with range c = 1. Each sensor outputs a value x E X that is 
an arbitrary function of the targets which it senses, x = ^({f v ■ v E S c ^}), where S Ct h is the 
coverage of a sensor located at grid block Fh with range c. Since the number of targets sensed by 
a sensor depends only on the sensor range, we write the number of targets in a sensor's coverage 
as |<S C |. We assume a simple model for randomly generating sensor networks, where each sensor 
chooses a region of Euclidean radius c with equal probability among the set of possible regions 
of radius c. This would occur, for example, if sensors were randomly dropped on a field. All 
definitions from the one-dimensional contiguous model extend directly, with target vectors v 
replaced by fields /. The rate is defined as R — —. 

For a sensor located randomly in the target field, the probability of a sensor producing a value 
depends on the number of target patterns that correspond to that value in the sensor's range, and 
thus, can be written as a function of the frequency of patterns in the field. The two-dimensional 
type 7j is a vector that corresponds to the normalized counts over the set of possible target 
configurations in the sensor's field of view in a field / i . For a sensor of range c, j i is a 2l 5c 
dimensional vector, where each entry in the vector j i corresponds to the frequency of occurrence 
of one of the possible |«S C | bit patterns. The set of sensor types 7 of a k x k field is denoted 
V%({0, l}' 5c '). 7(0) and 7(1) are the number of zeros and ones respectively in a vector of type 
7. These quantities can be directly computed from 7. 

Next, we note that for sensor of range c the conditional probability PxiXj depends on the 
two-dimensional joint type A of the i th and j th target fields fi, fj. For a,b E {0, l}l 5c l, A is the 
matrix of \ a )(b), the fraction of positions in fi, fj where fi has a target pattern a while fj has 
a target pattern b. We denote the set of all joint sensor types for sensors of range c observing 
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a target field of area k 2 , as V%({0, l}l 5c l, {0, l}l 5c l). Since the output of each sensor depends 
only on the contiguous region of targets which it senses, PxiXj depends only on A (discussed 
in Section IIV-BI) . Xnvo) is the number of grid locations where field i has a target and field j 
does not, and can be computed directly from A. A(o)n) is similarly defined and computed. 

Using the definitions of two dimensional types in the definitions of P x ]y an ^ QxiY from the 
one-dimensional contiguous connections model, we can prove the following bound for sensing 
a two-dimensional field. The sensing capacity at distortion D satisfies, 

C(D)>C LB (D)= min ' (2V) 

A (0)(l)+ A (l)(O)>- D 

where 7i , 7j . £ V 2 ({0, 1}^), 7i(0) = 0.5 and 7i(1) = 0.5, and A E V 2 ({0, 1}^, {0, 1}I 5 =I). 

Proof Outline: The proof is essentially identical to the proof of Theorem [Q with types and 
joint types replaced by two-dimensional types and joint types. For two-dimensional types, we 
bound a as follows, 

a(j u k) < 2 fc2 "«TWw» (28) 
For two-dimensional joint types, we bound (3 as, 

P{\,k) < 2 k2H{ ^)^w)) (29) 

The bounds on a and (3 are loose, and the authors are not aware of tighter combinatorial bounds 
for two-dimensional types. The set S^(D) is defined as in equation (|26|) . Given these new bounds 
and definitions, and the substitution of 2D types for types, the proof of Theorem Q] can be applied 
directly to prove this result. 

VI. Conclusions and Discussion 

The results presented in this paper provide limits on the accuracy of sensor networks for 
large-scale detection applications. These results are obtained by drawing on an analogy between 
channel coding and sensor networks. We define the sensing capacity and lower bound it for 
several sensor network models. For all rates below the sensing capacity, detection to within a 
desired accuracy with arbitrarily small error is achievable. This threshold behavior contrasts with 
classical detection problems, where probability of error goes to zero as the number of sensor 
measurements go to infinity while the number of hypotheses remains fixed [39]. The sensing 
capacity captures complex sensor tradeoffs. For example, our bounds show that the efficiency 
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of using long range, noisy sensors or shorter range, less noisy sensors depends on the desired 
detection accuracy. Further, our results show that the mutual information is not the correct notion 
of information for large-scale detection problems. This has implications for the problem of sensor 
selection due to the popularity of 'information gain' as a sensor selection metric. 

An important contribution of this paper is its demonstration of a close connection between 
sensor networks and communication channels. It is thought-provoking to consider that one 
could apply insights from the large body of work available for communication channels to the 
sensor network setting. For example, channel coding theory contains a large number of results 
that are used to build practical communication systems. Can we fruitfully apply ideas from 
coding theory to sensor networks? To demonstrate the potential benefit of a channel coding 
perspective, in [40], [41] we proposed extending ideas from convolutional coding to sensor 
networks. We demonstrated that a version of sequential decoding (a low complexity decoding 
heuristic for convolutional codes) can be applied to detection in sensor networks, as an alternative 
to the belief propagation algorithm. Our empirical results indicate that above a certain number 
of sensor measurements, the sequential decoding algorithm achieves accurate decoding with 
bounded computations per bit (target position). This empirical result suggests the existence of a 
'computational cut-off rate', similar to one that exists for channel codes. 

Our work on the theory of sensing points to a large set of open problems on large-scale detec- 
tion. Obvious directions include strengthening the theory by considering alternative settings of the 
problem, tightening the sensing capacity bounds, and proving a converse to sensing capacity. For 
example, we presented extensions to the work presented in this paper by considering the impact 
of spatial [38] and temporal [42] dependence on the sensing capacity. Another direction for 
future work is to explore the connection between sensor networks and communication channels, 
including the exploitation of existing channel codes to design sensor networks. 
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Fig. 1. Sensor network model. 
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Fig. 2. Communication channel model. 
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TABLE I 



Joint types A for four pairs of target vectors. 
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TABLE II 

Distribution of Xi in terms of the type 7 of vt when c = 2. 
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TABLE III 

Joint distribution of X, and Xi in terms of the joint type A of vi,vj when c = 2. 




Fig. 4. 



Clb(D) of arbitrary connections model for sensors of varying noise levels and range. 
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Fig. 5. Clb(O.I) of arbitrary connections model for sensors of varying noise levels, range, and sensing function. 




Fig. 6. Average empirical error rate of belief propagation based detection for varying rates, and the corresponding sensing 
capacity bound. 
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TABLE V 

Joint distribution of X, and Xi in terms of the joint type A of v } and vi, with c = 
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Fig. 8. Clb (0.025) for localized and non-localized sensors. 




Fig. 9. Sensor network model with k = 5, n = 2, c = 1. 



